2002

There are times when you feel you are witnessing a precise moment in which science fiction crosses over into science. I don't mean the advent of voice-recognition software that lets us talk to computers, or even the prospect of space tourism, but a moment when one of the great, enduring conundrums of our speculative literature suddenly materializes as a problem in actual life. Perhaps the greatest of these is the problem of distinguishing humans from robots. What element, if any, separates us from machines?

This theme received its classic treatment by Isaac Asimov in his 1946 story “Evidence.” In the story, a future society has forbidden the use of humanoid robots, since it is feared that the intelligent machines, identical in appearance to humans but with superior powers, will take over the world. A man named Stephen Byerley becomes the first “World Coordinator.” Many years later, however, his humanity is still in doubt.

“And the great Byerley was simply a robot.”

“Oh, there's no way of ever finding out. I think he was. But when he decided to die, he had himself atomized, so that there will never be any legal proof.—Besides, what difference would it make?”

What difference would it make? In January 2002, fifty-six years after the publication of Asimov's story, a group of computer scientists, cryptographers, mathematicians, and cognitive scientists gathered at “the first workshop on human interactive proofs.” Their goal was the creation of a CAPTCHA, a “Completely Automated Probabilistic Public Turing Test to Tell Computers and Humans Apart.”

In Asimov's story, distinguishing robots from humans was a matter of world-historical importance, a question of human dignity and worth. The problem for the scientists at the workshop appeared, on the surface, to be less consequential: the development of automated methods to prevent software robots, or “bots,” from invading chat rooms and barraging email systems with spam. Yet their work had implications beyond the needs of commercial applications. We have become dependent upon electronic communications; we can no longer envision a future without computer-mediated interactions; so the issue for us has become as dire as it was for the inhabitants of Asimov's imagined world. Their problem is now ours: How do we tell a human from a machine?

What is interesting about this problem is that it is one we humans have brought upon ourselves. It is a consequence of a deeply human part of us, Homo faber, the toolmaker. We have imagined the existence of robots, and, now that we have dreamed them up, it's as if we feel compelled to build them, and to endow them with as much intelligence as we can. We can't help it, it seems; it's in our nature as fashioners of helpful objects. We can't resist taking up the dare: How far can we go in creating smart tools until our tools come to outsmart us, ceasing in crucial ways to be “ours”?

Underlying that dare is a philosophical shift in science's views about the role of human beings in the great project of life. Researchers in robotics and artificial life openly question the “specialness” of human life. Some call life as we know it on earth merely one of many “possible biologies” and see our reverence for humanity as something of a prejudice (“human chauvinism”). According to Rodney Brooks, director of MIT's Artificial Intelligence Lab, evolution spelled the end of our uniqueness in relationship to other living creatures by defining us as evolved animals; and robotics, in its quest to create a sentient machine, looks forward to ending the idea of our uniqueness in relation to the inanimate world. In what may reflect supreme humility (we are no better than the rocks or the apes) or astounding hubris (we can create life without the participation of either God or the natural forces of evolution), computer science has begun the debate over the coming of what has come to be called the “post-human”: a nonbiological, sentient entity whose capabilities would exceed those of human beings.

According to this idea, the post-human's thoughts would not be limited by the slow speed of our own nervous systems. Unhampered by the messy wet chemistry of carbon-based life, loosed from the pressures of evolution, the post-human can be designed, consciously, to exceed our capabilities. Its memory can be practically limitless. It can have physical strength without bounds. And, freed from the senescence of the cells, it might live forever. If this sounds like Superman (“with powers far beyond those of mortal man”), consider another of those moments when science fiction passed over into science:

The date was April 1, 2000. The place was a lecture hall on the campus of Stanford University. Douglas Hofstadter, the computer scientist perhaps best known for his book Gödel, Escher, Bach, assembled a panel of roboticists, engineers, computer scientists, and technologists, and asked them to address the question: “Will spiritual robots replace humanity by 2100?”

Hofstadter began by saying, crankily, that he had “decided to eliminate naysayers” from the panel. He made his point with a cartoon of a fish that thinks it is ridiculous that life could exist on dry land (gribbit, gribbit, went the sound of a frog). “It is more amazing,” he said, “that life could come from inert matter than from a change of substrate”—more amazing that life could arise from a soup of dead molecules than change its base from carbon to something else, such as silicon. Hofstadter looked into the future, and said, without nostalgia or regret: “I really wonder whether there will be human beings.”

The room was filled to fire-marshal-alarming proportions. People jammed the doors, stood against the walls, sat in the aisles, on the steps in the steep balcony of the lecture hall, leaned dangerously against the balcony rails. The audience, young and old, students and veterans of the computing community of Silicon Valley, sat still and quiet, leaning forward, putting up with the crowding and the heat and Doug Hofstadter's grouchy refusal to use a microphone. From where I was sitting, high up in the balcony, the scene reminded me of nothing so much as those paintings of early medical dissections, crowds of men peering to where the cadaver lay slashed open in the operating theater below. On that day at Stanford, there was the same sense that some threshold, previously taboo to science, had been crossed. Computer science, which heretofore could have plausibly been said to serve humanity by creating its tools, was now considering another objective altogether: the creation of a nonbiological, “spiritual” being—sentient, intelligent, alive—who could surpass, and perhaps control, us.

*   *   *

This was not the first time that computer science seemed on the verge of creating a successor race of machines. When I was a young programmer, in the late 1970s and early 1980s, practitioners in a branch of computer science then called “artificial intelligence” believed they were close to creating an intelligent computer. Although AI would fail spectacularly in fulfilling its grand expectations at that time, the debate surrounding the field was alluring. I was then an in-the-trenches programmer, and, like many of my colleagues, I saw in AI the opportunity to explore the questions that had previously been in the province of the humanities. What are we? What makes a human intelligent? What is consciousness, knowledge, learning? How could these things be represented to a machine, and what would we learn about ourselves in the formation of that representation? It was soon clear that, in a secular society that had given up on the idea of God, we would be looking elsewhere for the answer to the question of what animates us; and that “elsewhere” would be the study of cybernetic intelligence, the engine of postmodern philosophical speculation.

It is for this reason that the question of the post-human is worth exploring. Whether or not we can build a “spiritual robot” by 2100, in asking what is “post”-human, we must first ask what is human. The ensuing debate inherits the questions that once belonged to philosophy and religion—and brings up the same ancient, deep-seated confusions.

Over the years, as I listened to the engineering give-and-take over the question of artificial life forms, I kept coming up against something obdurate inside myself, some stubborn resistance to the definition of “life” that was being promulgated. It seemed to me too mechanistic, too reductive of what we are. Even if I could not quite get myself to believe in God or the soul or the Tao, still, as I sat there high in the balcony of the Stanford lecture hall, listening to the cyberneticists' claims to be on the path toward creating a sentient being, I found myself muttering, “No, that's not right, we're not just mechanisms, you're missing something, there is something else, something more.” And then I had to ask myself: What else could there be?

*   *   *

Over the last half-century, in addressing the question “What are humans?” cybernetics has come up with three answers. We are, in order of their occurrence in the debate, (1) computers, (2) ants, and (3) accidents.

The first, the co-identification of human sentience and the computer, appeared almost simultaneously with the appearance of computers. In 1950, only four years after the construction of ENIAC, generally regarded as the first digital computer, the mathematician Alan Turing famously proposed that digital machines could think. And by the time computers had come into general use, in the 1960s, the view of the human brain as an information processor was already firmly installed.

It is an odd view, if you consider it. ENIAC was conceived as a giant calculator: it was designed to compute the trajectory of artillery shells. Its role was understood as human complement, doing well what we do poorly (tedious computation, precise recall of lists of numbers and letters) and doing badly what we do well (intuitive thinking, acute perception, reactions involving the complex interplay of mental, physical, and emotional states). Yet, by 1969, when computers were still room-sized, heat-generating behemoths with block-letter-character screens, the computer scientist and Nobel Laureate in economics Herbert Simon felt no compunction about grouping computers and humans as related entities: “The computer is a member of an important family of artifacts called symbol systems … Another important member of the family (some of us think, anthropomorphically, it is the most important) is the human mind and brain.” Simon went on to postulate “if computers are organized somewhat in the image of man”—without going on to question that “if.” In existence barely twenty-five years, the machine that was designed to be our other—the not-human, accurate in a world where to be human is to err—had become the very analogue of human intelligence: the image of man.

*   *   *

Herbert Simon, along with his colleague Allen Newell, was a pioneer in the field of artificial intelligence. It is worthwhile now to give a close reading to Simon's groundbreaking book The Sciences of the Artificial, written in 1969. For here one can see the origins of the curious reasoning whereby the computer becomes a model for humanity.

Simon begins by discussing what on the surface might seem obvious: the difference between the natural and the artificial worlds. Natural objects have the authority of existence, he says; the “laws” of nature determine what must be. The artificial, in contrast, is designed or composed. “The engineer, and more generally the designer, is concerned with how things ought to be—how they ought to be in order to attain goals, and to function.”

Then Simon's definition of an artifact becomes more complex. He describes it not as a thing in itself but as an interface. The artificial object is an interaction between “the substance and organization of the artifact itself, and an ‘outer' environment, the surroundings in which it operates.” The artifact, then, does not exist on its own. It is a bodiless, abstract process mediating between the article's substance and its existence in the world.

That is fair enough. Even a stone moves in reaction to a shifting environment. But then Simon's reasoning takes an odd turn. He goes on to say: “Notice that this way of viewing artifacts applies equally well to many things that are not man-made—to all things in fact that can be regarded as adapted to some situation; and in particular it applies to the living systems that have evolved through the forces of organic evolution.” That is, it applies to all creatures—and, by extension, to us.

By the sixth page of Simon's book, where these statements appear, human beings have been removed from the realm of the “natural.” Viewed as adaptable products of evolution, we have become hollow artifacts, interfaces to our environment, “systems” engineered by the processes of natural selection. It is a startling turnabout: what is being proposed here is not the possibility of creating artificial life but the redefinition of life itself as artificial.

*   *   *

Once you accept the definition of human life as artificial—designed, engineered—it is then an easy matter to say that the proper study of man is not man but some other engineered object, the machine. And this is indeed what Simon advocates: making the computer itself the object of study, as a phenomenon of a living system. “Since there are now many such devices in the world [computers], and since the properties that describe them also appear to be shared by the human central nervous system, nothing prevents us from developing a natural history of them. We can study them as we would rabbits or chipmunks and discover how they behave under different patterns of environmental stimulation.” Standing amazed before this human-created machine, the computer scientist declares it to be our very identity; then, to learn who and what we are, he advises that we study … the machine.

This circular idea—the mind is like a computer; study the computer to learn about the mind—has infected decades of thinking in the computer and cognitive sciences. We find it in the work of Marvin Minsky, the influential figure in artificial intelligence, who, when asked if machines could think, famously answered: “Of course machines can think; we can think and we are ‘meat machines.'” We see it in the writings of the cognitive scientist Daniel Dennett, whose book Consciousness Explained is suffused with conflations between human sentience and computers: “What counts as the ‘program' when we talk of the virtual machine running on the brain's parallel hardware?… And how do these programs of millions of neural connection-strengths get installed on the brain's computer?” And we can read it in the extreme predictions made by the engineer Ray Kurzweil, inventor of the Kurzweil music synthesizer and of reading systems for the blind. Kurzweil sees the coming of “spiritual robots” almost entirely in the language of computer programming: calling memory “the mind file,” which he will scan and “download” to a silicon substrate, analyzing it for its basic “algorithms,” thereby creating a “backup copy” of the original human being, all without the aid of natural evolution, which he calls “a very inefficient programmer.”

The limitations of this model of human intelligence should have become clear with the fading of that first naïve blush of optimism about the creation of sentient machines. In selecting the computer as the model of human thinking, AI researchers were addressing only one small portion of the mind: rational thought. They were, in essence, attempting a simulation of the neocortex—rule-based, conscious thinking—declaring that to be the essence of intelligence.

*   *   *

And AI did succeed in creating programs that relied upon rule-based thinking, creating so-called expert systems that codified narrow, specific, expert domains, such as oil exploration and chess playing.1 But, as was pointed out by Hubert Dreyfus, the early critic of AI, the results of AI were disappointing, systems devoid of presence and awareness, a “disturbing failure to produce even the hint of a system with the flexibility of a six-month-old child.”

Yet the idea of human being as computer lives on. Most troubling, even after becoming controversial in computer science, it has taken residence in the natural sciences. As Rodney Brooks put it in an article in Nature: “The current scientific view of living things is that they are machines whose components are biochemicals.” The psychologist Steven Pinker, in the first pages of his classic book How the Mind Works, writes that problems of understanding the human “are both design specs for a robot and the subject matter of psychology.” And this view shows up in surprising places—for instance, in an email I received from Lucia Jacobs, a professor of psychology who studies squirrel behavior at Berkeley: “I am an ethologist and know virtually nothing about computers, simulations, programming, mathematical concepts or logic. But the research is pulling me right into the middle of it.”

Herbert Simon's views have come full circle; it is now standard scientific practice to study machine simulations as if they were indeed chipmunks, or squirrels: “What seems to be crystallizing, in short,” wrote Jacobs of her work with robotic creatures, “is a powerful outlook on spatial navigation, from robots to human reasoning. This is wonderful.” Psychology and cognitive science—and, indeed, biology—are poised to become, in essence, branches of cybernetics.

However, the view of human biology as purely cybernetic may not endure indefinitely. In February 2001, the government-funded Human Genome Project and the private company Celera Genomics announced that they had succeeded in sequencing the full complement of human genes. What they found was shocking from a scientific point of view. “There are far fewer genes than we expected, only about 30,000 or so rather than the figure of 100,000 in the textbooks,” said Dr. Eric Lander, who headed the Human Genome Project. “There is a lesson in humility in this. We only have twice as many genes as a fruit fly or a lowly nematode worm. What a comedown.” And there went the simple mechanical view of one gene making one protein, and then—chunk, chunk—a machine processing all the proteins, and out comes one person.

Researchers are looking for other ways to explain the complexity of human physical and mental existence, said Lander. Perhaps the complexity comes from the way proteins are folded. Perhaps some proteins have multiple roles. Perhaps some genes are not “expressed,” do not function in some circumstances. Perhaps there are some unknown relationships among the genes, tangles of interactions that will forever lead us down and down into the nature of matter itself. But one thing is clear: DNA is not “code.” Now that we know the complete human genome, DNA has literally been decoded, stripped of its programming powers.

*   *   *

Failing to produce intelligence by modeling the “higher functions” of the cortex, cybernetics next turned to a model creature without any cortex at all: the ant. This seems like an odd place to look for human intelligence. Ants are not generally thought of as smart. But as a research model they provide one enormous advantage over the study of human brains: they yield an explanation of how apparent complexity can arise without an overseeing designer. A group of dumb ants produces the complexity of the ant colony—an example of organizational intelligence without recourse to the perennial difficulties of God or philosophy.

Again, the source for this key idea seems to be Herbert Simon. The third chapter of his book The Sciences of the Artificial opens by describing an ant making its way across a beach:

We watch an ant make his laborious way across a wind- and wave-molded beach. He moves ahead, angles to the right to ease his climb up a steep dunelet, detours around a pebble, stops for a moment to exchange information with a compatriot. Thus he makes his weaving, halting way back to his home. So as not to anthropomorphize about his purposes, I sketch the path on a piece of paper. It is a sequence of irregular, angular segments—not quite a random walk, for it has an underlying sense of direction, of aiming toward a goal.

I show the unlabeled sketch to a friend. Whose path is it? An expert skier, perhaps, slaloming down a steep and somewhat rocky slope. Or a sloop, beating upwind in a channel dotted with islands or shoals. Perhaps it is a path in a more abstract space: the course of search of a student seeking the proof of a theorem in geometry.

The ant leaves behind it a complex geometric pattern—how? The ant has not designed this geometry. Simon's revolutionary idea was to locate the source of the complexity not in the ant, which is quite simple “viewed as a behaving system,” but in the ant's interaction with its environment: in the byplay of the ant's simple, unaware reactions to complications of pebble and sand. Simon then goes on to pronounce what will turn out to be inspirational words in the history of cybernetics:

In this chapter I should like to explore this hypothesis but with the word “human being” substituted for “ant.”

With that, Simon introduces an idea that will reverberate for decades across the literature of robotics and artificial life. One can hardly read about the subject, or talk to a researcher, without coming upon the example of the ant—or the bee, or termite, or swarm, or some other such reference to the insect world. It is not clear if later adopters of “the ant idea” completely grasp the implications of Simon's view. They have concentrated upon the many low-level, dumb interactions of ants as they exchange pheromones, leaving out the difficulties of each creature's interplay with the environment. Yet what was distilled out of Simon's utterance has become an enduring model: organizational intricacy arises spontaneously out of the normal workings of the individual participants in a society or system.

This phenomenon, known as “emergence,” produces outcomes that cannot be predicted from looking only at the underlying simple interactions. This is the key idea in fields known, variously, as “complexity theory,” “chaos theory,” and “cellular automata.” It is also a foundation concept in robotics and “ALife,” a branch of computer science that concerns itself with the creation of software exhibiting the properties of life. “Emergence” lets researchers attempt to create intelligence from the bottom up, as it were, starting not from any theory of the brain as a whole, but from the lowest levels of the body. The idea seems to be that, if you construct a sufficient number of low-level, atomic interactions (“automata”), what will eventually emerge is intelligence—an ant colony in the mind.

Sentience is not a thing, according to this view, but something that arises from the organization of matter itself. The roboticist Hans Moravec writes:

Ancient thinkers theorized that the animating principle that separated the living from the dead was a special kind of substance, a spirit. In the last century biology, mathematics, and related sciences have gathered powerful evidence that the animating principle is not a substance, but a very particular, very complex organization. Such organization was once found only in biological matter, but is now slowly appearing in our most complex machines.

In short, given enough computing power—which gets easier every year, as the computational abilities of chips increase exponentially—it's possible to build a robotic creature which crosses some critical threshold in the number of low-level organizational interactions it is able to sustain. Out of which will emerge—like Venus surfacing from the sea on the half shell—sentience.

There is a great flaw in this reasoning, however. Machines are indeed getting more and more powerful, as was predicted by the former Intel chairman Gordon Moore, in what has come to be known as Moore's Law. But computers are not just chips; they also need the instructions that tell the chips what to do: the software. And there is no Moore's Law for software. On the contrary, as systems increase in complexity, it becomes harder—very much harder—to write reliable code.

In a quest for robust systems, software engineers turned to working with what are called object-oriented methods. In this model, code is written in very small chunks, each performing the most minimal of tasks. The objects send small “messages” to one another, like ants rubbing antennas to communicate via pheromones. For example, when you ask a program to print a document, you first get a “dialog” object, which calls upon an “input characters” object and a “click button” object, another object that interacts with the printer, and so on. This is a highly simplified example. Something as mundane as printing evokes perhaps hundreds of objects, all of which run within a universe of tens of thousands of objects.

Roboticists foresee code that “evolves,” objects that succeed in a process of Darwinian selection, eventually creating a self-sustaining system. The best of the objects are invoked for years; they are the hearty survivors. Weaker code is replaced and then must also stand up to the test of time. In this view, code objects are automata; as their numbers rise into the millions and billions, a self-organizing intelligence will emerge.

That thick cloud of interacting objects—that atmospheric mist of code—does indeed begin to run as if on its own. The complexity of the environment tends to move beyond the easy comprehension of an individual programmer. But what happens in this “running on its own” is a system soon running amok. It crashes, fails, succumbs to bugs. It is up to the human programmer to intervene: understand it, fix it, change it.

Darwinian selection, therefore, cannot be invoked as a rationale for the emergence of cybernetic life. Evolution is predicated upon the dire human imperatives to survive and reproduce. We are not abstract processes; we are not Simon's bodiless artifacts. We do not face survival pressures mindlessly. We know we are not ants. We know we are not code objects. We know we are going to die. In the quest to build humanoid robots, researchers may say they are building something—a mechanical object that simulates some aspects of human sentience. But it remains a simulation. The organizational principles of that mechanical object do not illuminate the real bases of human sentience. They do not tell us what we are.

*   *   *

The mistake in robotics is the same as that in AI: mistaking the tool for its builder. In particular, the error comes from mistaking the current methods of software writing as a paradigm for human mental organization. In the 1970s, a computer program was a centralized, monolithic thing, a small world unto itself, a set of instructions operating upon a set of data. It should be no surprise, then, that researchers at the time saw the human intelligence as a … centralized, monolithic, logical mind operating upon a set of data. By the 1990s, that monolithic paradigm of programming had been replaced by the object-oriented methods described above, with code written in discrete, atomic chunks that could be combined in a variety of ways. And—what do you know?—human sentience was hence seen as something emerging from the complex interaction of … discrete atomic chunks. Is cognitive science driving the science of computing, or is it the other way around?

There are signs that even the cyberneticists who promoted the concept of emergence as a paradigm for human sentience are sensing its limits. Christopher Langton, a key figure in ALife research, admitted there is the problem of “finding the automata”—deciding what indeed constitutes the lowest-level interactions that must be simulated in order to create life. How deep must one go: to interactions between cells? molecules? atoms? elementary particles of matter? Talking with me at a café in Linz, Austria, Langton looked up from a scribbled notebook and said with sincere worry: “Where's the bottom of physics?”

Meanwhile, the roboticist Rodney Brooks wondered about the “top” of the problem, the higher-level cognitive functions that are supposed to emerge from an organism's underlying complexity. After years spent creating robots that were like insects, Brooks recognized that something else is involved in the grand project of intelligence. He is now revisiting the problem that stumped AI researchers in the 1970s: how to give a cybernetic creature an awareness of its own state of being—its own needs, desires, intentions—and also an understanding that others have their own states of being—needs, desires, intentions—that are different from its own. In a discussion we held in March 2001, Brooks looked like a man who knew he was stepping off the brink when he said: “We're trying to introduce a theory of mind.”

The bottom of physics. A theory of mind. And here we go again. Back we are drawn into the metaphysical thickets from which engineering hoped permanently to flee. The hope was to turn sentience into a problem not of philosophy but of engineering. “You don't have to understand thought to make a mind,” said Douglas Hofstadter, wishfully, while introducing the spiritual-robot panel at Stanford. “The definition of life is hard,” Rodney Brooks said to me. “You could spend five hundred years thinking about it or spend a few years doing it.”

And here is the underlying motive of robotics: an anti-intellectualism in search of the intellect, a flight from introspection, the desire to banish the horrid muddle of all this “thinking about it,” thousands of years of philosophical speculation about what animates us, without notable progress. “You can understand humans either by reverse engineering or through building,” said Cynthia Breazeal, a former student of Brooks's. Don't think about it; build it—that's the hope. Equate programming with knowledge. Yet still we circle back to the old confusions, for conceptualization is as deep in our human nature as is tool building. Homo faber wrestles with Homo sapiens.

*   *   *

One way to get around the difficulties of human sentience is to declare humans all but irrelevant to the definition of life. This is the approach taken by ALife researchers, who see human beings—indeed, all life on earth—as “accidents,” part of the “highly accidental set of entities that nature happened to leave around for us to study.” As Christopher Langton writes in his introduction to Artificial Life: An Overview, “The set of biological entities provided to us by nature, broad and diverse as it is, is dominated by accident and historical contingency … We sense that the evolutionary trajectory that did in fact occur on earth is just one out of a vast ensemble of possible evolutionary trajectories…”

Based on the same foundations as modern robotics—emergence theory—ALife's goal is the creation of software programs that exhibit the properties of being alive, what is called “synthetic biology,” the idea being that researchers can learn more about life “in principle” if they free themselves of the specific conditions that gave rise to it on earth. ALife research says farewell to the entire natural world, the what-must-be in Herbert Simon's formulation, with barely a backward glance (except occasionally to cite the example of ants).

“Life” in the context of ALife is defined very simply and abstractly. Here are two typical approaches: “My private list [of the properties of life] contains only two items: self-replication and open-ended evolution,” writes Thomas S. Ray. And “life must have something to do with functional properties we call adaptive, even though we don't yet know what those are,” writes Stevan Harnad. Bruce Blumberg, an MIT researcher who creates robotic dogs in software animations, describes the stance of ALife this way: “Work has been done without reference to the world. It's hard to get students to look at phenomena. It's artificial life, but people aren't looking at life.”

What ALife researchers create are computer programs—not robots, not machines, only software. The cybernetic creatures in these programs (“agents” or “automata”) go on to “reproduce” and “adapt.” They thereby fulfill the basic requirements that allow them to be seen as living entities. So does the image of the computer as human paradigm, begun in the 1950s, come to its logical extreme: pure software, unsullied by exigencies of carbon atoms, bodies, fuel, gravity, heat, or any other messy concern of either soft-tissued or metal-bodied creatures. Again the image of the computer is conflated with the idea of being alive, until only the computer remains: life that exists only in the machine.

*   *   *

What these views of human sentience have in common, and why they fail to describe us, is a certain disdain for the body: the utter lack of a body in early AI and in later formulations like Kurzweil's (the lonely cortex, scanned and downloaded, a brain-in-a-jar); and the disregard for this body, this mammalian flesh, in robotics and ALife.

Early researchers were straightforward about discarding the flesh. “Meat machines,” Marvin Minsky pronounced us to be. Said Herbert Simon: “Instead of trying to consider the ‘whole person,' fully equipped with glands and viscera, I should like to limit the discussion to Homo sapiens, ‘thinking person.'” Meat and glands and viscera—you can sense the corruption implied here, the body as butchery fodder, polluting the discussion of intelligence.

This suspicion of the flesh, this quest for a disembodied intelligence, persists today. Ray Kurzweil brushes aside the physical life as irrelevant to the project of building “spiritual” beings: “Mammalian neurons are marvelous creations, but we wouldn't build them the same way. Much of their complexity is devoted to supporting their own life processes, not to their information-handling abilities.” In his view, “life” and “information-handling” are not synonymous; indeed, “life” gets in the way. He sees evolution as “a sloppy programmer,” producing DNA that is mostly “useless” and “replete with redundancies.”

And ALife researchers, seeing “life” in their computer programs, pay no attention at all to the body, imagining that the properties of life can somehow, like tissue specimens, be cut away from the dross of living. As Thomas Ray writes:

Whether we consider a system living because it exhibits some property that is unique to life amounts to a semantic issue. What is more important is that we recognize that it's possible to create disembodied but genuine instances of specific properties of life in artificial systems. This capability is a powerful research tool. By separating the property of life that we choose to study from the many other complexities of natural living systems, we make it easier to manipulate and observe the property of interest. [Emphasis added.]

One might think that robotics, having the imperative of creating some sort of physical container for intelligence, would have more regard for the human body. But the entire project of robotics—the engineering of intelligent machines—is predicated upon the belief that sentience is separable from its original substrate.

I had a talk with Cynthia Breazeal, who is now on the faculty of the MIT Media Lab. She is a most thoughtful researcher. Breazeal's work involves the creation of a creature that responds to human beings with simulated emotional reactions, and she shows a sincere regard for the emotional life. Yet even she revealed an underlying distaste for the body—a characterization she disagreed with. Growing impatient with me as I pressed her for a definition of “alive,” she said: “Do you have to go to the bathroom and eat to be alive?”

The question stayed with me—Do you have to go to the bathroom and eat to be alive?—because it seemed to me that Breazeal's intent was to cite the most basic acts required by human bodily existence, and then see them as ridiculous, even humiliating.

But after a while I came to the conclusion: Maybe yes. Given the amount of time living creatures devote to food and its attendant states—food! the stuff that sustains us—I decided that, yes, there might be something crucial about the necessities of eating and eliminating that defines us. How much of our state of being is dependent upon being hungry, eating, having eaten, being full, shitting. Hunger! Our word for everything from nourishment to passionate desire. Satisfied! Meaning everything from well fed to sexually fulfilled to mentally soothed. Shit! Our word for human waste and an expletive of impatience. The more I thought about it, the more I decided that huge swaths of existence would be impenetrable—indescribable, unprogrammable, utterly unable to be represented—to a creature that did not eat or shit.

In this sense, artificial-life researchers are as body-loathing as any medieval theologian. They seek to separate the “principles” of life and sentience—the spirit—from the dirty muck from which it sprang. As Breazeal put it, they envision a “set of animate qualities that have nothing to do with reproduction and going to the bathroom,” as if these messy experiences of alimentation and birth, these deepest biological imperatives—stay alive, eat, create others who will stay alive—were not the foundation, indeed, the source, of intelligence; as if intelligence were not simply one of the many strategies that evolved to serve the creatural striving for life. If sentience doesn't come from the body's desire to live (and not just from any physical body, but from this body's striving, this particular one), where else would it come from? To believe that sentience can arise from anywhere else—machines, software, things with no fear of death—is to believe, ipso facto, in the separability of mind and matter, flesh and spirit, body and soul.

Here is what I think: Sentience is the crest of the body, not its crown. It is integral to the substrate from which it arose, not something that can be taken off and placed elsewhere. We drag along inside us the brains of reptiles, the tails of tadpoles, the DNA of fungi and mice; our cells are permuted paramecia; our salty blood is what's left of our birth in the sea. Genetically, we are barely more than roundworms. Evolution, dismissed as a sloppy programmer, has seen fit to create us as a wild amalgam of everything that came before us: except for the realm of insects, the whole history of life on earth is inscribed within our bodies. And who is to say which piece of this history can be excised, separated, deemed “useless” as an essential part of our nature and being?

*   *   *

George Lakoff, the linguist, is perhaps the thinker most responsible for bringing the body back into the discussion of human intelligence. He and the philosopher Mark Johnson are the co-authors of two remarkable books (Metaphors We Live By and Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought) that outline the ways in which the body is integral to intelligence and human consciousness. Thinking is not logical and conscious, they argue, but metaphorical and mostly unconscious; that is, unavailable to rational introspection. And the source of most of these metaphors is the body itself. “The body provides the metaphors for the world,” Lakoff said while speaking to a Berkeley group called the Philosophical Club. “This is not solipsistic, because we have evolved these bodies to interact with the world, as it is.” Even formal logic itself has its source in physical existence: “The fact that we have muscles and use them to apply force in certain ways leads to the structure of our systems of causal concepts.” Lakoff's most recent book (with Rafael Núñez, Where Mathematics Comes From) goes so far as to posit the idea that even mathematics, that branch of abstract thinking normally considered the polar opposite of the flesh, is also an extension of the body's “existence in space.” As Lakoff and Johnson wrote in Philosophy in the Flesh:

Reason is not disembodied, as the tradition has largely held, but arises from the nature of our brains, bodies, and bodily experience. This is not just the innocuous and obvious claim that we need a body to reason; rather, it is the striking claim that the very structure of reason itself comes from the details of our embodiment.

The details of our embodiment: the complex of tissues, fluids, sinew, and bone that we are made of. If we were made of something else—integrated circuits, for instance—we might have something called logic, but it would not necessarily be analogous to the ways human beings interpret the world—indeed, might not be anything we would recognize as intelligence at all. As Lakoff put it at the meeting, “What would be the logic of a bat? A jellyfish?”

*   *   *

Intelligence, then, is a consequence of our having this particular fleshly form, and what I mean by this particular form goes beyond that of human beings, or even primates. What I'm referring to is our existence as mammals. Oddly, in all the views of human intelligence promulgated by cyberneticists, this is the one rarely heralded: what we call sentience is a property of mammalian life.

Mammalian life is social and relational. What defines the mammalian class, physiologically, is not dependence upon the female mammary gland. What makes mammals different from other animals is the possession of a portion of the brain known as the limbic system. And it is the limbic system that allows us to do what other animals cannot: read the interior state of others of our kind.

To survive, we need to know our own inner state and those of others, quickly at a glance, deeply. This is the “something” we see in the eyes of another mammalian creature: it looks at us and knows, in its own way, that we have feelings, states, desires that are different from its own. We look at each other and know we are separate beings who nonetheless communicate. This is what people mean when they say they can talk to their dogs or cats, horses or bunnies: mammals reading each other. We don't go looking for this in ants or fish or reptiles; indeed, when we want to say that someone lacks that essential spark of life, we call them “reptilian.” What we mean by this is that they lack emotions, the ability to relay and read the emotions of others; that they are, in short, robotic.

If sentience is a mammalian trait, and what distinguishes mammals is the capacity for social life, then sentience must have its root in the capacity for rich social and emotional interchange. That is: Sentience begins with social life, with the ability of two creatures to transact their inner states—needs, desires, motivations, fears, threats, contentment, suffering, what we call “the emotions.” Moreover, the more avenues a creature has for understanding and expressing its emotional states, the more intelligent we say it is. Ants were not a good place to look for rich social interchange; the logical inference engines of early AI were a particularly poor choice of model; computer software running in the astringent purity of a machine won't find it. To get at the heart of intelligence, we should have started by looking at the part of human life ordinarily considered “irrational,” the opposite of “logical,” that perennial problem for computers: emotions.

*   *   *

The role of emotions in sentience can be seen in something as deceptively simple as a chair. Hubert Dreyfus pointed out the futility of trying to create a symbolic representation of a chair to an object that had neither body to sit in it nor a social context in which to use it: “What makes an object a chair is its function, and what makes possible its role as equipment for sitting is its place in the total practical context. This supposes certain facts about human beings (fatigue, the way the body bends) … Chairs would not be equipment for sitting if our knees bent backwards like those of flamingos, or if we had no tables, as in traditional Japan or the Australian bush.”

The chair makes no sense without the emotions that accompany sitting down: relief for someone standing for a long while; boredom, edginess, anxiety for someone too long seated; feelings associated with what we are doing while sitting in the chair, hours at work on a deadline or the enervation of hours staring at a television. These and any number of other remembered and emotional states are part of our understanding of this object: schoolroom hard bench, mother's ugly armchair, dirty thing you retrieved from a curbside to take into your first apartment. We can construct an abstract idea of a chair—describe its use, say what it's made of, formulate its geometry, describe its relationship to other furniture; we can fashion a robot that moves its mechanical limbs in such a way that it “sits”—but none of this approaches the sheer richness of the sentient human experience that surrounds and suffuses this object, an experience simultaneously mental, tactile, physical, emotional, social, and cultural, each of these aspects so intertwined as to be inseparable.

*   *   *

Some roboticists are beginning an investigation into the ways in which a mechanical object can have, or appear to have, an emotional and social existence. “Most roboticists couldn't care less about emotions,” says Cynthia Breazeal, one of the few who do care about emotions. Her “Kismet” robot, a very cute device with the face and floppy ears of a bunny rabbit, has simulated emotional states (expressed adorably, ears drooping piteously when it's sad); it is designed to interact with and learn from humans as would a child. “Social intelligence uses the whole brain,” she says. “It is not devoid of motivation, not devoid of emotion. We're not cold inference machines. Emotions are critical to our rational thinking.”

Rodney Brooks speaks of adding “an emotional model,” of giving his new robot “an understanding of other people.” Cynthia Breazeal's Kismet robot is designed to interact with humans and to “suffer” if it doesn't get attention. Bruce Blumberg, who “does dogs,” as he puts it, understands that “you can't say you're modeling dogs without social behavior.”

Of the three, however, only Blumberg seems to grasp the enormity of the problem they're undertaking: he is willing to admit that there is something ineffable about a living being's social and emotional existence. “My approach is to build computer devices to catch a spark of what's really there in the creature,” he says, “to understand what makes dogs—and us—have a sort of magical quality.” Then, perhaps embarrassed at this recourse to magic, he adds, “Ninety-nine percent of computer scientists would say you're no computer scientist if you were talking in terms of ‘magical qualities.'”

Indeed, his colleague Breazeal has a pragmatic, somewhat cynical view of the emotions. Robots will need to have something like emotions, she says, because corporations are now investing heavily in robotic research, and “emotions become critical for people to interact with robots—or you won't sell many of them.” The point seems to be to fool humans. About her robot Kismet she says, “We're trying to play the same game that human infants are playing. They learn because they solicit reactions from adults.”

But an infant's need for attention is not simply a “game.” There is a true, internal reality that precedes the child's interchange with an adult, an actual inner state that is being communicated. An infant's need for a mother's care is dire, a physical imperative, a question of life or death. It goes beyond the requirement for food; an infant must learn from adults to survive in the world. But without a body at risk, in a creature who cannot die, are the programming routines Breazeal has given Kismet even analogous to human emotions? Can a creature whose flesh can't hurt feel fear? Can it suffer?

Even if we leave aside the question of embodiment; even if we agree to sail away from the philosophical shoals of what it really means to have an emotion as opposed to just appearing to have one—the question remains: How close are these researchers to constructing even a rich simulation of mammalian emotional and social life?

Further away than they realize, I think. The more these researchers talked about their work, the longer grew the list of thorny questions they know they will have to address. “Is social behavior simply an elaboration of the individual?” asked Blumberg. “What does the personality really mean?” “We need a model of motivation and desires.” “How much of life is like that—projection?” From Breazeal: “How do you build a system that builds its own mind through experience?” And this great conundrum: “A creature needs a self for social intelligence—what the hell is that?”

In turning to the emotions and social life, they have hit right up against what Breazeal calls the “limiting factor: big ideas”—theories of learning, brain development, the personality, social interaction, motivations, desires, the self—essentially, the whole of neurology, physiology, psychology, sociology, anthropology, and just a bit of philosophy. It all reminded me of the sweet engineering naiveté of Marvin Minsky, back in the early days of AI, when he offhandedly suggested the field would need to learn something about the nature of common sense. “We need a serious epistemological research effort in this area,” he said, believing it would be accomplished shortly.

*   *   *

Of course, the biggest of the “big ideas” is that old bugaboo consciousness. Being difficult, fuzzy, unwilling to yield up its secrets despite thousands of years devoted to studying it, consciousness is something robotics researchers would rather not discuss. “In our group, we call it the C-word,” said Rodney Brooks.

Brooks is an urbane and charming man, with a soft Australian accent. He seems genuinely interested in exchanging thoughts about arcane matters of human existence. It was early March 2001; outside was the beginning of a snowstorm that was predicted to shut down all of the Boston area. Brooks sat with me at a small conference table in his office at MIT, where photographs of his insectlike robots hung on the walls, and, piled in the corner among some books, was the robotic doll called “My Real Baby” he had made for the Hasbro toy company.

Consciousness, of course, is a problem for robots. Besides its being hard to simulate, the very idea of consciousness implies something unfathomably unique about each individual, that “magical quality” Bruce Blumberg was daring enough to mention. Brooks's impulse, like that of his former student Cynthia Breazeal, was to view the interior life rather cynically, as a game, a bunch of foolery designed to elicit a response.

I mentioned Breazeal's Kismet, told him I thought it was designed to play on human emotions. Then I asked him, “Are we just a set of tricks?”

He answered immediately. “I think so. I think you're a bunch of tricks and I'm just a bunch of tricks.”

Trickery is deeply embedded in the fabric of computer science. The test of machine intelligence that Alan Turing proposed in 1950, now known as the Turing Test, was all about fooling the human. The idea was this: Have a human interact with what might be either a computer or another human being. Place a person behind a curtain, able to see the text of the responses but not who or what “said” them. If the person cannot tell if the response comes from another human being or from a machine, then the machine will be judged to be humanly intelligent. A circus stunt, if you will. A Wizard of Oz game. A trick.

Just then, sitting in Brooks's office, I didn't much feel like a bunch of tricks. I didn't want to think of myself as what Brooks had just described as “just molecules, positions, velocity, physics, properties—and nothing else.” He would say this was my reluctance to give up my “specialness”; he would remind me that it was hard at first for humans to accept they descended from apes. He would not understand that I was not trying to define humans, as a species, as anything “special.” Only that each individual human, ape, chimp, dog—any creature with a “theory of mind”—is special unto itself.

But I was aware of something else in me protesting this idea of the empty person, nothing but a set of mechanisms, a zombie process. It was the same sensation I'd had while at the spiritual-robot symposium hosted by Douglas Hofstadter, an internal round-and-round—the clicking, unbalanced ceiling-fan of doubt—that kept me thinking, No, no, no, that's not it, you're missing something.

I asked Brooks if he knew what consciousness was.

He answered, “I don't know. Do you know what consciousness is good for?”

And without hesitation, I told him yes, I knew what consciousness is good for. I told him we are born helpless and defenseless. Our only hope to survive is to make contact with other humans. We must learn to tell one individual from another, make alliances, immediately see on the face of another human being whether this is friend or foe, parent or stranger, kin or enemy. I told him I think human existence as a species is predicated on this web of social interactions, and for this we must learn to identify individuals. And out of that, the recognition of the identity of others, comes our own identity, the sense that we exist, ourselves, our self. Everything we call consciousness unwinds from that.

“It's not mystical,” I told him. “It's an evolutionary imperative, a matter of life and death.”

Brooks put his chin on his hand and stared at me for a moment.

Then he said: “Huh. None of our robots can recognize their own kind.”

*   *   *

It took me several months, but after thinking about Rodney Brooks's remark about robots and their own kind, my round-and-round anxiety—that voice in me that kept protesting, No, no, you're missing something—finally stopped. For there it was, the answer I had been looking for, the missing something else: recognition of our own kind.

This is the “magical quality,” mutual recognition, the moment when two creatures recognize each other from among all others. This is what we call “presence” in another creature: the fact that it knows us, and knows we know it in turn. If that other being were just a trick, just the product of a set of mechanisms, you would think that snakes could make this recognition, or paramecia, or lizards, or fish. Their bodies are full of marvelous mechanisms, reflexes, sensors to give them an awareness of the world around them. Ant pheromones should work. Robots with transponders beaming out their serial numbers should do the job. But we are, as Cynthia Breazeal said, creatures whose brains are formed by learning—that is, through experience and social interaction. We don't merely send out signals to identify ourselves; we create each other's identities.

It is true that identity, the idea of the human being as a unity, is not an entirely accurate concept. As Lakoff pointed out, most of our intelligence is unconscious, not available for introspection, having an independent existence, so to speak. And the body itself is not a unity, being instead a complicated colony of cells and symbiotic creatures. We can't live without bacteria in our gut; tiny creatures live on our skin and eyelids; viruses have incorporated themselves into our cells. We're walking zoos. Yet somehow, for our own survival (and pleasure), it is critical that we attain a unified view of ourselves, as unique selves.

This idea of being a unique self is not just some chauvinistic sense of specialness, some ego problem we have to let go of. Nature has gone to a great deal of trouble to make creatures within the same species distinct from one another. The chromosomes mix themselves up in the reproductive cells. Through the wonder of naturally recombinant DNA, nearly every human being on earth is distinct from every other. This recombining of the genetic material is usually thought of as creating diversity, but the corollary effect is the creation of uniqueness. Identical twins fascinate us for this reason: because they are rare, the only humans on earth (along with triplets, quintuplets, and other identical siblings) without their own faces. We're born distinct, and as our brains develop in the light of experience, we grow ever more different from one another. Mammalian life takes advantage of this fact, basing our survival on our ability to tell one from another, on forming societies based on those mutual recognitions. Uniqueness, individuality, specialness are inherent to our strategy for living.

AI researchers who are looking at social life are certainly on the right path in the search to understand sentience. But until they grasp the centrality of identity, I don't think they'll find what they're looking for. And then, of course, even supposing they grant there is something called an identity, a unique constellation of body and experience that somehow makes a creature a someone, a self—“What the hell is that?” said Cynthia Breazeal—even so, they will still have to find a way to program it.

Their task in simulating a self-identifying sentient creature will be like trying to simulate a hurricane. I thought about how weather simulations work. Unable to take into account all the complexity that goes into the production of weather (essentially, the whole world), simulations use some subset of that complexity and are able to do a fairly good job of predicting what will happen in the next hours or days. But as you move out in time, or at the extremes of weather, the model breaks down. At three days, the understanding gets chancy. Ten days out, the simulation doesn't work. The fiercer the storm, the less the simulation knows what it will do. Hurricanes are not something you predict; they're something you watch. And that's what human sentience is: a hurricane—too complex to understand fully by rational means, something you observe, marvel at, fear with a sense of awe, what finally we give up and call “an act of God.”